Skip to main content

View Post [edit]

Poster: ReadingRainbowSponsoredPizza Date: Jan 19, 2017 5:20pm
Forum: software Subject: Torrents? Padding files are a problem

First of all t_m_c, thank you for the MAME2003 reference set. It is amazingly well done.

My issue is that it looks like the archive was made with BitComet which is the only client that generates things called padding files on creation. This padding is causing close to a a third of the total space to be wasted in both bandwidth and disk space for clients. In the day and age of severely limited bandwidth caps, this just hurts everyone.

It was and is still, sadly, used by bitcomet to pad out each file so each file fits in it's own piece of the torrent. This is so you would not have to rely on another unavailable file to build another file. This is not bad with an a single PDF or the piece size is not large, but is a real problem for an archive with thousands of files about 1-3 megs big where the piece size is 4 megs. In simpler terms, if there is a 1 meg file, a padding file (consisiting of nothing but nulls) 3 megs big will be made to match it.

For example, the MAME2003_Reference_Set_MAME0.78_ROMs_CHDs_Samples archive, the rom directory is 29.5 GBs big. The padding file directory (.___padding_file) is 14.7 GBs. Someone using a torrent client will see the total download size for this archive would be 40 gigs which can be confusing since the archive states it is only about 30 gigs large.

edit: looking at the torrent download itself, I see it's made by "ia_make_torrent" not BitComet like I initially stated. Is there a document of why this position of creating empty padding files was taken up?



This post was modified by ReadingRainbowSponsoredPizza on 2017-01-20 01:20:57

Reply [edit]

Poster: Jeff Kaplan Date: Jan 20, 2017 1:00pm
Forum: software Subject: Re: Torrents? Padding files are a problem

from our engineer:
"We generate our own Torrents with our own in-house software; padding is an intentional feature (sic) which allows us to update Torrents when item files change without re-hashing the entire item.

Idle comment, the naming convention used was dictated by BitTorrent Inc. and in theory is a de facto standard; some clients at the time of our software writing anyway would ignore such padding files so long as the convention was followed.

We do apologize for the overhead this causes, we will take this under consideration!"